course evaluation
An Exploration of Higher Education Course Evaluation by Large Language Models
Course evaluation is a critical component in higher education pedagogy. It not only serves to identify limitations in existing course designs and provide a basis for curricular innovation, but also to offer quantitative insights for university administrative decision-making. Traditional evaluation methods, primarily comprising student surveys, instructor self-assessments, and expert reviews, often encounter challenges, including inherent subjectivity, feedback delays, inefficiencies, and limitations in addressing innovative teaching approaches. Recent advancements in large language models (LLMs) within artificial intelligence (AI) present promising new avenues for enhancing course evaluation processes. This study explores the application of LLMs in automated course evaluation from multiple perspectives and conducts rigorous experiments across 100 courses at a major university in China. The findings indicate that: (1) LLMs can be an effective tool for course evaluation; (2) their effectiveness is contingent upon appropriate fine-tuning and prompt engineering; and (3) LLM-generated evaluation results demonstrate a notable level of rationality and interpretability.
- North America > United States (0.14)
- Asia > China > Beijing > Beijing (0.04)
- Oceania > Australia > Queensland > Brisbane (0.04)
- North America > Canada > British Columbia > Metro Vancouver Regional District > Vancouver (0.04)
- Instructional Material > Course Syllabus & Notes (1.00)
- Research Report > New Finding (0.66)
Leveraging Large Language Models for Actionable Course Evaluation Student Feedback to Lecturers
Zhang, Mike, Lindsay, Euan D, Thorbensen, Frederik Bode, Poulsen, Danny Bøgsted, Bjerva, Johannes
End of semester student evaluations of teaching are the dominant mechanism for providing feedback to academics on their teaching practice. For large classes, however, the volume of feedback makes these tools impractical for this purpose. This paper explores the use of open-source generative AI to synthesise factual, actionable and appropriate summaries of student feedback from these survey responses. In our setup, we have 742 student responses ranging over 75 courses in a Computer Science department. For each course, we synthesise a summary of the course evaluations and actionable items for the instructor. Our results reveal a promising avenue for enhancing teaching practices in the classroom setting. Our contribution lies in demonstrating the feasibility of using generative AI to produce insightful feedback for teachers, thus providing a cost-effective means to support educators' development. Overall, our work highlights the possibility of using generative AI to produce factual, actionable, and appropriate feedback for teachers in the classroom setting.
- Europe > Denmark > North Jutland > Aalborg (0.06)
- Europe > Denmark > Capital Region > Copenhagen (0.05)
- Overview (1.00)
- Instructional Material (1.00)
- Research Report > New Finding (0.66)
- Education > Educational Setting > Online (0.69)
- Education > Educational Technology > Educational Software > Computer Based Training (0.46)
Detecting Gender Bias in Course Evaluations
Lindau, Sarah, Nilsson, Linnea
We use different methods to examine and explore the data and find differences in what students write about courses depending on gender of the examiner. Data from English and Swedish courses are evaluated and compared, in order to capture more nuance in the gender bias that might be found. Here we present the results from the work so far, but this is an ongoing project and there is more work to do.
- Research Report > New Finding (0.73)
- Research Report > Experimental Study (0.73)
An Analysis of Programming Course Evaluations Before and After the Introduction of an Autograder
Hagerer, Gerhard Johann, Lahesoo, Laura, Anschütz, Miriam, Krusche, Stephan, Groh, Georg
Commonly, introductory programming courses in higher education institutions have hundreds of participating students eager to learn to program. The manual effort for reviewing the submitted source code and for providing feedback can no longer be managed. Manually reviewing the submitted homework can be subjective and unfair, particularly if many tutors are responsible for grading. Different autograders can help in this situation; however, there is a lack of knowledge about how autograders can impact students' overall perception of programming classes and teaching. This is relevant for course organizers and institutions to keep their programming courses attractive while coping with increasing students. This paper studies the answers to the standardized university evaluation questionnaires of multiple large-scale foundational computer science courses which recently introduced autograding. The differences before and after this intervention are analyzed. By incorporating additional observations, we hypothesize how the autograder might have contributed to the significant changes in the data, such as, improved interactions between tutors and students, improved overall course quality, improved learning success, increased time spent, and reduced difficulty. This qualitative study aims to provide hypotheses for future research to define and conduct quantitative surveys and data analysis. The autograder technology can be validated as a teaching method to improve student satisfaction with programming courses.
- Europe > Germany > Bavaria > Upper Bavaria > Munich (0.05)
- North America > United States > New York > New York County > New York City (0.04)
- North America > United States > Hawaii (0.04)
- (3 more...)
- Research Report > Experimental Study (1.00)
- Instructional Material > Course Syllabus & Notes (1.00)
- Research Report > New Finding (0.94)
- Education > Educational Technology > Educational Software (1.00)
- Education > Educational Setting > Online (0.93)
- Education > Curriculum > Subject-Specific Education (0.67)
What Learning Can Learn From Machine Learning
Over the years, this biweekly letter has provided me with the opportunity to fully and fairly document just how much free time college students can have if they try. My college roommates tried really hard. They found time to make prank calls to the campus literary magazine, create enough frost in our fridge to throw snowballs out the window on 90-degree days, leave old pizza in the entryway for the stated purpose of growing penicillin for a roommate who couldn't afford antibiotics, and organize campus recruiting events for fake investment banks. When these time-wasting activities required a fake identity, the persona of choice was John W. Moussach Jr., an alumnus turned successful Midwestern industrialist. Last week I looked online for remnants of John W. Moussach Jr. and came upon neither the Wikipedia page my roommates built after graduating nor the Moussach aphorism that somehow made it onto Wikiquote ("We have all heard the Will Rogers quote'I never met a man I did not like.' In my youth, I met a World War I veteran who had met Will Rogers. The veteran told me, 'I never met a man I did not like until I met Will Rogers'"), but rather an article on something called Study Sive which purports to feature higher education news.